1. Field of Invention
The present invention relates to the field of matrix factorization. More specifically, it relates to the field of kernel nonnegative matrix factorization, kernel NMF, with incorporated data classification properties.
2. Description of Related Art
Nonnegative matrix factorization (NMF) has recently been used for various applications, such as face recognition, multimedia, text mining, and gene expression discovery. NMF is a part-based representation wherein nonnegative inputs are represented by additive combinations of nonnegative bases. The inherent nonnegativity constraint in NMF leads to improved physical interpretation compared to other factorization methods, such as Principal Component Analysis (PCA). Psychological and physiological evidence for part-based representations in the brain appear to support the advantage of NMF.
Interestingly, NMF does not explain the recognition capability of the brain since NMF and its variants have been designed not for classification but for reconstruction. The lack of classification capability is a natural consequence of the unsupervised factorization method that does not utilize relationships within the input entities, such as class labels.
Several approaches have been proposed for NMF to generate more descriptive features for classification and clustering tasks. For example, “Fisher Nonnegative Matrix Factorization”, ACCV, 2004, by Y. Wang, Y. Jiar, C. Hu, and M. Turk, proposes incorporating the NMF cost function and the difference of the between-class scatter from the within-class scatter. However, the objective of this Fisher-NMF is not guaranteed to converge since it may not be a convex function. “Non-negative Matrix Factorization on Manifold”, ICDM, 2008, by D. Cai, X. He, X. Wu, and J. Han proposes graph regularized NMF (GNMF), which appends the term representing the favorite relationships among feature vector pairs. But, GNMF is handicapped by not considering unfavorable relationships.
Recently, J. Yang, S. Yang, Y. Fu, X. Li, and T. Huang proposed “Non-negative graph embedding” (NGE), in CVPR, 2008. NGE resolved the previous problems by introducing the concept of complementary space so as to be widely considered the state-of-the-art. NGE, however, utilized the approximate formulation of graph embedding, and as a result, NGE is not effective enough for classification, particularly when intra-class variations are large. This limitation is highlighted in experimental results shown below.
In a general sense, all of these previous works tried to incorporate NMF with graph embedding, but none of them successfully adopted the original formulation of graph embedding because the incorporated optimization problem is intractable. In addition, all the works are limited in that they depend on suitable parameters which are not easy to be determined appropriately.